Summary

Adds a max_page parameter to chunk_by_title() that gives callers a hard upper bound on how many pages a single chunk may span. max_page is a strict superset of the existing multipage_sections flag: max_page=None (default) matches multipage_sections=True, and max_page=1 matches multipage_sections=False — plus the new ability to express any page-count limit (max_page=2, max_page=3, …) that the old binary flag could not.

multipage_sections is kept for backward compatibility but deprecated. It emits a DeprecationWarning and will be removed in a future release.

Motivation

chunk_by_title supported character/token-based size limits and an all-or-nothing page boundary toggle (multipage_sections). There was no way to say "allow cross-page sections, but cap each chunk at N pages". This is a common need in RAG pipelines where page-level context windows matter — e.g. a retriever that attaches page images to each chunk should not pull more than a fixed number of page images per chunk.

Behaviour

When max_page=N is specified, the chunker tracks the page where each chunk began and starts a new chunk as soon as an element's page number is N or more pages past that start page:

chunks = chunk_by_title(elements, max_page=2)
# → each chunk spans at most 2 pages

chunks = chunk_by_title(elements, max_page=1)
# → a new chunk starts on every page change

• Additive constraint — works alongside all existing by_title boundaries (title, size, token).
• Title-aware reset — a Title element resets the page counter without double-firing, so each title-delimited section's page span is measured independently.
• Hard limit, not a hint — PreChunk.can_combine() also enforces max_page, so PreChunkCombiner cannot silently re-merge pre-chunks that were split by the page boundary.
• No page-number, no problem — elements without metadata.page_number are assumed to continue the current page.
• Validation — max_page < 1 raises ValueError.

multipage_sections deprecation

multipage_sections is kept but deprecated. Existing call sites continue to work without modification; they will receive a DeprecationWarning pointing to the max_page equivalent.

When both multipage_sections and max_page are provided, max_page takes priority silently — no error is raised.

Example

from unstructured.chunking.title import chunk_by_title
from unstructured.documents.elements import Title, Text, ElementMetadata

elements = [
    Title("Introduction",  metadata=ElementMetadata(page_number=1)),
    Text("Page 1 content.", metadata=ElementMetadata(page_number=1)),
    Text("Page 2 content.", metadata=ElementMetadata(page_number=2)),
    Text("Page 3 content.", metadata=ElementMetadata(page_number=3)),
    Text("Page 4 content.", metadata=ElementMetadata(page_number=4)),
    Text("Page 5 content.", metadata=ElementMetadata(page_number=5)),
]

chunks = chunk_by_title(elements, max_page=2, combine_text_under_n_chars=0)
# chunk 0 → pages 1-2  (Introduction + Page 1 + Page 2)
# chunk 1 → pages 3-4  (Page 3 + Page 4)
# chunk 2 → page  5    (Page 5)

Bug fix: PreChunkCombiner was undoing max_page boundaries

Root cause: PreChunker correctly split elements using is_on_page_exceeding_max, but PreChunkCombiner only checked combine_text_under_n_chars and hard_max when deciding whether to merge consecutive pre-chunks. Two pre-chunks split by a max_page boundary could be silently re-merged if their combined text was small enough, making max_page a soft hint rather than a hard limit.

Fix:

• Added a max_page hook to ChunkingOptions base (returns None; overridden by _ByTitleChunkingOptions) so PreChunk.can_combine() can enforce the limit without coupling to a specific strategy.
• Added PreChunk._page_span — a cached property returning (first_page, last_page) for elements that carry a page number.
• Guarded PreChunk.can_combine() to block combination whenever the merged span would exceed max_page pages.

Changes

unstructured/chunking/base.py

• New is_on_page_exceeding_max(max_page) boundary predicate factory — handles all page-boundary cases including max_page=1.
• ChunkingOptions.max_page — new cached property returning None by default; overridden by _ByTitleChunkingOptions.
• PreChunk._page_span — cached property returning (first_page, last_page).
• PreChunk.can_combine() — page-span guard blocks combination that would violate max_page.

unstructured/chunking/title.py

• chunk_by_title() gains max_page: Optional[int] = None. multipage_sections is kept but emits DeprecationWarning when passed.
• _ByTitleChunkingOptions.max_page — reads max_page kwarg directly; falls back to translating multipage_sections=False → 1 when max_page is not set. max_page takes priority when both are provided.
• _ByTitleChunkingOptions.multipage_sections — kept as a read-only backward-compat alias (max_page != 1).
• _ByTitleChunkingOptions.boundary_predicates — uses is_title plus is_on_page_exceeding_max when max_page is set; is_on_next_page() removed.
• _ByTitleChunkingOptions._validate() — rejects max_page < 1.

unstructured/chunking/dispatch.py

• Help text updated to advertise max_page (by_title only) and mark multipage_sections as deprecated.

Structural diff

%%{init: {'layout': 'elk', 'elk': {'direction': 'RIGHT'}, 'maxTextSize': 999999, 'theme': 'base', 'themeVariables': {'background': '#ffffff', 'clusterBkg': '#f8fafc', 'clusterBorder': '#94a3b8', 'primaryColor': '#f8fafc', 'primaryBorderColor': '#94a3b8', 'primaryTextColor': '#1e293b', 'lineColor': '#64748b', 'fontSize': '13px', 'fontFamily': 'ui-monospace, SFMono-Regular, Menlo, monospace'}}}%%
classDiagram
    direction LR

    class n2["_ByTitleChunkingOptions"] {
        <<unstructured/chunking/title.py>>
        + max_page()
        ~ _validate()
        ~ boundary_predicates()
        ~ iter_boundary_predicates()
        ~ multipage_sections()
    }
    class n5["unstructured/chunking/title.py"] {
        ~ chunk_by_title()  sig
    }

    class n0["ChunkingOptions"] {
        <<unstructured/chunking/base.py>>
        + max_page()
    }
    class n1["PreChunk"] {
        <<unstructured/chunking/base.py>>
        + _page_span()
        ~ can_combine()  calls +1
    }
    class n3["unstructured/chunking/base.py"] {
        + is_on_page_exceeding_max()
        + page_count_exceeded()
    }

    style n2 fill:#fefce8,color:#854d0e,stroke:#f59e0b,stroke-width:3px
    style n5 fill:#fefce8,color:#854d0e,stroke:#f59e0b,stroke-width:3px
    style n0 fill:#f0fdf4,color:#166534,stroke:#22c55e,stroke-width:3px
    style n1 fill:#fefce8,color:#854d0e,stroke:#f59e0b,stroke-width:3px
    style n3 fill:#f0fdf4,color:#166534,stroke:#22c55e,stroke-width:3px

    n2 --|> n0
    n2 --> n3 : calls

• Green — new (ChunkingOptions.max_page, is_on_page_exceeding_max, page_count_exceeded)
• Yellow — modified (_ByTitleChunkingOptions, PreChunk, chunk_by_title signature, dispatch.py)
• _ByTitleChunkingOptions --|> ChunkingOptions — inherits and overrides max_page
• _ByTitleChunkingOptions --> base.py — calls is_on_page_exceeding_max; is_on_next_page removed
• multipage_sections() remains on _ByTitleChunkingOptions as a read-only backward-compat alias

Testing

pytest test_unstructured/chunking/test_title.py -q
# 60 passed, 7 skipped

New / updated tests:

• test_max_page_1_is_equivalent_to_multipage_sections_false — proves max_page=1 and the deprecated multipage_sections=False produce identical output
• test_chunk_by_title_respects_max_page — chunks split at correct page boundaries
• test_chunk_by_title_max_page_resets_on_title — Title resets the counter independently per section
• test_chunk_by_title_max_page_1_breaks_on_every_page — max_page=1 breaks on every page change
• test_chunk_by_title_max_page_not_undone_by_combiner — regression: combiner cannot re-merge max_page-split pre-chunks
• test_chunk_by_title_max_page_none_does_not_add_page_boundary — omitting max_page leaves existing behaviour unchanged
• it_translates_multipage_sections_false_to_max_page_1 — internal translation verified
• it_leaves_max_page_none_when_multipage_sections_is_true — no-op case verified
• it_prefers_max_page_over_multipage_sections_when_both_are_set — priority rule verified
• it_emits_deprecation_warning_when_multipage_sections_is_used — warning verified
• it_does_not_warn_when_multipage_sections_is_not_passed — no spurious warnings
• it_accepts_valid_max_page_values — parametrized: 1, 3, None
• it_rejects_max_page_less_than_one — parametrized: 0, -1, -10
• Existing multipage_sections integration tests wrapped with pytest.warns(DeprecationWarning)

Checklist

• New functions have docstrings and type hints
• Tests added and passing (60 passed, 7 skipped)
• CHANGELOG.md updated
• __version__.py bumped to 0.23.4
• No debugging artifacts or commented-out code